Training Bi-Encoders for Word Sense Disambiguation

Modern transformer-based neural architectures yield impressive results in nearly every NLP task and Word Sense Disambiguation, the problem of discerning correct sense a word given context, is no exception. State-of-the-art approaches WSD today leverage lexical information along with pre-trained embeddings from these models to achieve comparable human inter-annotator agreement on standard evaluation benchmarks. In same vein, we experiment several strategies optimize bi-encoders for this specific propose alternative methods presenting our model. Through multi-stage pre-training fine-tuning pipeline further state art Disambiguation.